Method and system for testing variations of website content

ABSTRACT

The invention is a method and system for testing variations of website content useful to optimize website visitor conversions. The invention helps website operators create content variations, and then scientifically test and measure the impact of those variations on conversion rates. The invention includes a method of using a reverse proxy server to introduce page variations on existing website content without needing to modify the existing web server. Using the reverse proxy server, the invention automatically injects values, associated with website variations, into web page tags used with web analytics platforms. The invention makes it possible to test any number of website variations and access such data from within web analytics platforms. The invention also provides a method to automatically transform web page tags of a first web analytics platform to web page tags of a second web analytics platform to easily try new third party web analytics solutions.

CROSS-REFERENCE TO RELATED APPLICATIONS

None.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever. Copyright 2007 SiteSpect, Inc.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to online marketing, advertising, andweb functionality. The invention relates more specifically to conversionmarketing, website testing, and web analytics.

2. Background

Website operators are challenged to maximize conversion rates—the numberof visitors who make a purchase, register for a new account, sign up fora newsletter, submit a sales lead, adopt a website feature, etc. Awebsite's conversion rate is a measure of how successfully a websitepersuades visitors to act on what is offered.

The conversion process is normally comprised of numerous pages andlinks. The content on each page (including the text headings of eachlink) must compel the visitor forward through the site. While a fewvisitors quickly move forward through a site, the majority of visitorsnever click to a second page. Of the few that do click to a second page,only a small percent follow through with any conversion action.

To optimize web conversions, the website operator must determine howbest to speak to the site's visitors, in a way that visitors understand,telling visitors what they need to know at each point in the conversionprocess. Specifically, the website operator needs to test variations ofwebsite content (including, but not limited to, product descriptions,headlines, calls to action, assurance messages, images, and words thataid site navigation and usability) and determine which are mostpersuasive to visitors.

Conversion marketing is the science of measuring, testing, and improvingthe rate at which website visitors respond to website content with apredefined action or actions, including, but not limited to, clicking alink, filling out a web page form, putting an item in a shopping basket,subscribing to a newsletter, spending time consuming specific content,or purchasing a product. More background on conversion marketing andwebsite content testing is found in applicant's US Patent ApplicationPublication Number 2006/0271671, filed Nov. 30, 2006, titled “System AndMethod For Optimizing Website Visitor Actions,” which is hereinincorporated by reference.

Web analytics relates to measuring and analyzing the actions andbehavior of visitors to a website. Web analytics primarily focuses onbusiness aspects of a visitor's actions such as clicking on advertisinglinks, making purchases, consuming content, and requesting information.Web analytics platforms are software programs or web services formanaging, tracking, and reporting data associated with visitor actions.There are many different actions and data that web analytics platformscan track and report. Such data or actions may include: requests forpage views, requests for files on a web server a series of requests froma single visitor, whether visitors are new or repeat visitors,advertisement impressions, and bounce rates,

DESCRIPTION OF PRIOR ART

There have been several attempts at optimizing web conversions usingvariation testing. One model for such testing is a remote content servermodel, which may be operated either by a third party vendor or by thesite operator itself. In such a model, a request from a visitor's webbrowser is sent directly to a website. The website responds withcustomized content that includes JavaScript code. The JavaScript codesubsequently contacts a remote server of the optimization tool vendor.The web browser executes the JavaScript and requests variable web pagecontent from a remote server. Then the remote server processes the webbrowser's JavaScript request, sending the web browser content variationsand tracking the visitor's session, typically using a cookie. The webbrowser inserts the content variations into the web page to completerendering.

While the remote content server model provides a relatively simple wayto introduce variations into multiple regions on the same page, themodel suffers from a number of disadvantages:

1) The website operator must modify a web server's website pages toinclude the JavaScript code.

2) The web browser must have JavaScript enabled.

3) The web browser may have to make multiple HTTP connections to theremote server, which slows overall page load time.

4) This model cannot track or modify non-HTML content, such as “.TXT,”“.XML,” “.EXE,” or “.PDF” files since JavaScript is not executed forthose document types.

5) Since the page is not fully rendered until processed by theend-user's browser, the content that the user sees cannot be recordedfor auditing or analysis purposes (which is done by products such asTealeaf so that the operator's technical staff can investigate problemsreported by end users).

6) The model can typically only insert static content into a page(whether the encompassing page itself is statically or dynamicallygenerated). This prevents testing variations of the dynamic contentitself, such as the formatting of a catalog page or search engineresults.

Another model for testing variations is a remote redirection servermodel. In this model, a visitor clicks on a static link (HREF) for awebsite, and the web browser makes a request to a redirect server orredirect page. The redirect server responds with an HTTP “Location”redirect to a link on the target website. The redirect server alsotracks a visitor's session, typically using a cookie. The web browseraccepts the redirect and automatically requests a new URL from thewebsite. The website then responds with content variations.

The remote redirection server model improves on the remote contentserver model in that it does not require JavaScript in the web browser.Although the remote redirection server model is able to track clicks andprovide variations of any type of content, the model suffers from anumber of disadvantages:

1) A website operator must physically create new variations of pages,and must re-code links wherever the resulting page may require avariation.

2) The web browser has to make two requests for a test page (one to theredirect server, and a second for the redirect to the website), whichslows overall page load time.

3) Page content cannot be dynamically altered by the testing product.

4) Links may display the URL of the redirect server instead of that ofthe website.

5) The web browser may display a warning when redirecting from HTTPS toHTTP sites (or vice versa).

Another model for testing variations is an integrated module model. Inthis model, a visitor's web browser requests content directly from atarget website. The target website server is integrated with a marketingoptimization module so that the visitor's request is handled within thesame system. The integrated marketing module instructs the website howto render content, track actions, and so forth. The website serverreturns content variations directly to the web browser.

The integrated model uses the website platform's own session managementfacility and thus can avoid using additional cookies and JavaScript. Byintegrating the marketing module with the website platform, there arerelatively fewer time delays, more content variation options within thesame page, and the model can track clicks and provide variations of anytype of content. The integrated model, however, suffers from a number ofdrawbacks:

1) Installing an integrated model is complex, expensive, and timeconsuming to the website operator.

2) The website operator must reconfigure the website architecture to becompatible with the marketing vendor's integrated module.

3) The integrated module cannot manage marketing tests that spanheterogeneous servers. For example, if a website operator runs avendor's integrated module on a first subdomain (e.g. www1.example.com),but not on a second subdomain of the website (e.g. www2.example.com),then only the first subdomain can use the marketing test module, eventhough the two systems may host the website jointly.

There are also a number of patents and patent applications that attemptto optimize web conversions.

U.S. Patent Application Publication Number 2006/0200556 “Method AndApparatus For Identifying, Extracting, Capturing, And LeveragingExpertise And Knowledge” (Brave, Sep. 7, 2006) discloses a method todetermine knowledge within a community of users. The objective is toimprove content searches, but does not measure or optimize user behaviorfrom the perspective of the site operator.

U.S. Patent Application Publication Number 2002/0099818 “Method AndSystem For Monitoring The Performance Of A Distributed Application”(Russell, Jul. 25, 2002) discloses a method of determining one or moreperformance metrics for a distributed application. The objective ofRussell is to measure and track system/network level performance metricssuch as pageload latency/time/speed, javascript errors on the page, etc.Russell discloses inserting tags to accomplish performance metricsmonitoring. Russell addresses how well the end user's computer/browseris able to communicate with a web server.

U.S. Patent Application Publication Number 2006/0200503 “ModifyingBack-End Web Server Documents At An intermediary Server UsingDirectives.” (Dosa, Sep. 7, 2006) discloses a method of providingnetwork services by a network coupled back-end server that is accessedby an intermediary server such as a reverse proxy server. Dosa providesa solution to allow cell phones to access external data source whenresponding to a Web service request.

U.S. Patent Application Publication Number 2004/0177318 “IdentifyingStatements Requiring Additional Processing When Forwarding A Web PageDescription,” (Raja, Sep. 9, 2004) discloses a system to enable anadministrator to conveniently specify statements of a web pagedescription which may require additional processing. When statements ofweb page descriptions potentially require additional processing, areverse proxy server is used to compare statements of web pagedescriptions with rules to determine if additional processing is needed.

U.S. Pat. No. 5,968,125 “Process For Optimizing The Effectiveness Of AHypertext Element” (Garrick et al. Oct. 19, 1999) discloses a processfor determining the effectiveness of a web page to a visitor by creatingalternative and test web pages, sending requests to the test web page,and monitoring the use of the web page and the rate that the web pageobjective was met and replacing pages with the page most visited.

U.S. Patent Application Publication Number 20020042738 “Method AndApparatus For Determining The Effectiveness Of Internet Advertising”(Srinivasan et al. Apr. 11, 2002) discloses a method and system formeasuring the effectiveness of the layout or appearance of a websiteadvertisement to a visitor wherein different visitors are showndifferent formats of the same page and response to the page is monitoredand statistics are analyzed regarding the responses.

U.S. Patent Application Publication Number 20030014304 “Method OfAnalyzing Internet Advertising Effects” (Calvert et al. Jan. 16, 2003)discloses a method of evaluating Internet advertisement effectivenessthat involves collecting Internet activity information associated with amultitude of cookies.

U.S. Patent Application Publication Number 20030018501 “Adaptive TestingFor Conversion-Related Estimates Relevant To A Network Accessible Site”(Shan Jan. 23, 2003) discloses a method and system for processing testdata relevant to specific behavior of visitors of a website.

U.S. Pat. No. 6,662,215 “System And Method For Content Optimization”(Moskowitz et al. Dec. 9, 2003) discloses a system and method fordetermining appropriate website content for consumers comprising aserver arrangement, including a “real time content optimization” server,a user computer, and a network, wherein upon request a web page isgenerated for the user having static and dynamic elements which aretested for user reaction and response.

U.S. Patent Application Publication Number 20040030597 “Method AndSystem Of Optimizing The Response And Profitability Of A MarketingProgram” (Bibas Feb. 12, 2004) discloses a method and system fortracking a consumer's behavior on a website, dynamically changing anadvertisement or sales page, and testing a sales page to a subset of themarketing program to optimize the content of the page.

U.S. Patent Application Publication Number 20040123247 “Method andapparatus for dynamically altering electronic content” (Wachen, Mark etat. Jun. 24, 2004) discloses “[a] method and apparatus for alteringelectronic content includes a template for assigning variables andvalues to a section of the content, a generator that creates thepermutations of the content, a transmitter that provides the content toa requestor and a evaluator and optimizer that aids in selecting themost optimal permutation of the content.”

While the prior art discloses attempts to improve online marketing, noneof the web conversion optimization models provides a method ofintroducing page variations on existing website content without needingto make modifications to existing, back-end, content on the web server.What is needed, therefore is

1) a product that does not require modifications to an existing webserver,

2) a product that automatically makes website content variations,

3) a product that can introduce variations on both static and dynamicwebsite content, and

4) a website testing product that can be integrated with existingthird-party web analytics platforms.

BRIEF SUMMARY OF THE INVENTION

The invention is a web software product that helps website operatorscreate content variations, scientifically test and measure the impactthat those content variations have on visitor conversion actions, andintegrate data from content variations with web analytics platforms. Theinvention makes it possible to test any number of website variations—inany combination—on website visitors. The invention produces conclusiveanswers about the impact of any type of content change on visitorconversion rates. Website operators learn, for example, which productdescriptions yield the most sales, which promotional language pulls themost sales leads, and what site registration form generates the mostsign-ups.

The invention provides a method of using a reverse proxy server tointroduce page variations on existing website content, without the needto make any modification to existing (back-end) content on the webserver or target web server. The web server hosts a website to be testedand evaluated. The reverse proxy server delivers web content variationsto visitors. A visitor's web browser—or any electronic device or programcapable of accessing web content—requests website content located at theweb server. A reverse proxy server receives this request for websitecontent. The reverse proxy server retrieves website content from the webserver. The web server sends unmodified content to the reverse proxyserver. The reverse proxy server modifies website content and transmitsthe modified content to a requester, such as a visitor or a browser. Thereverse proxy server tracks a visitor's actions (actions includingresponses, non-responses, conversions, non-conversions, etc.).

The invention integrates test campaign data with web analytics platformsincluding third-party web analytics platforms (e.g. such as thoseprovide by Google Analytics, WebSideStory, Omniture, Coremetrics, etc.).The invention feeds or supplies test campaign data to web analyticplatforms by injecting values into web page tags, page forms, URL's andlinks, or cookie values. Web page tagging is a method of collecting webanalytics data by using executable scripts on each web page to notify athird-party server when a page is rendered by a web browser. Forexample, if a site uses WebSideStory HBX for web analytics, then thatsite will have existing JavaScript tags on each web page. The inventionwill then insert its own data values into the tags' custom variables sothat the results of multivariate comparison tests may be tracked andanalyzed within the WebSideStory HBX reporting system.

The invention alternately inserts new third-party JavaScript tags onto apage where they do not already exist. This provides a turnkey mechanismfor implementing third party web analytics solutions. Instead of theprior art approach of having to modify an underlying site to implementpage tagging, the invention automatically inserts such tags into webpages. The invention can conditionally insert different tags (or tagvalues) based on parameters such as URL, domain name, whether or notspecific content exists on the page, and so on.

The invention alternately transforms third-party web page tags. Theinvention alters or morphs an existing third-party vendor's analyticstag into that of yet another vendor. This is useful in situations wherea website operator wishes to try a new web analytics provider and/ormigrate away from an existing provider. The invention looks for anexisting JavaScript tag, parses out page-specific values, then replacesthe tag—or appends a new tag—with the parsed page-specific values. Thebenefit is that the site operator is no longer locked into an existinganalytics solutions due to the effort required to remove the old tagsand add the new tags and the potential data tracking/collection lossduring the interim period where there are no tags on the site. Siteoperators are now free to try new vendors and migrate their third-partyanalytics solutions using the invention as the enabler.

In one embodiment, the method of testing variations of website contentincludes receiving a request at a reverse proxy server for websitecontent located at a web server, that is the website content is locatedat the web server. The website content can be any static or dynamiccontent in various formats (text, images, animations, audio, etc.). Theweb server is the target of the request and is where website content islocated. The reverse proxy server sits in front of the web server. Thatis, connections coming from the Internet addressed to the web server arerouted through the reverse proxy server. The reverse proxy serverretrieves requested website content from the web server. This websitecontent contains a web page tag associated with a web analyticsplatform. The web page can use any scripting language, but preferablyuses JavaScript. The reverse proxy server, or a module on the reverseproxy server, modifies website content by introducing variations in thewebsite content. At the reverse proxy server, the invention also insertsvalues, associated with website content variations for testing, into theweb page tag. By inserting website testing variation data into pagetags, data from website content variations is accessible within areporting system of a web analytics platform, including third-partyplatforms. The modified website content, having web pages tags linked tovariation data, is transmitted to a requester which is a websitevisitor, a browser, crawler program, or the like.

In another embodiment, the invention creates new web page analytics tagsat the reverse proxy server and inserts these created tags into modifiedwebsite content. This is for situations where a website to be testedcontains no web page tagging linked to web analytics platforms. The webpage analytics tags that the invention creates and inserts are designedfor, or formatted for, tracking with web analytics reporting systems.

In another embodiment, at the reverse proxy server, the inventiontransforms or modifies existing web page tagging designed for use at afirst web analytics platform so that modified website content containsweb page tagging formatted for tracking at a second web analyticsplatform. The invention alternately replaces an existing tag, or appendsa new tag.

The reverse proxy server may track a visitor's session with its owncookie, the web server's cookie, a third-party web analytics system'scookie, using the web server's URL/path translation to piggyback on theweb server's session management facilities, using a hidden form field,or by any other method of session management. The system may also keep avisitor's session through the reverse proxy servers session managementfacilities, independent of the web server. The system on the reverseproxy server evaluates requests and responses and tracks a visitorsactions. The modification module modifies content to introducevariations as required, and responds to the web browser with finalcontent.

The web server website operator creates test campaigns to measure theimpact of variations of copy, headlines, positioning, layout, imagery,price, etc. The modification module includes a user interface forallowing the website operator to create site variations using a WYSIWYG(what-you-see-is-what-you-get) visual editor. This eliminates the needto modify the web server content and saves time and cost otherwise spenton web designers and programmers. The operator can also targetoptimization efforts by segmenting visitors by specific criteria. Theoperator can stipulate that one or more visitor segments are assigned ornot assigned to a test group based on operator-controlled parameters,assignment criteria, and test campaign parameters.

The testing is then automated through the reverse proxy server system.Testing includes use of A/B testing, split path (i.e., split run)testing, and multivariate testing. During the test campaign, themodification module collects data for each response point (action) thatthe website operator desires to measure. Some measurable actionsinclude: adding an item to a shopping cart, registering for an account,and completing a sale.

At any time during a test campaign, the website operator can run areport to see how variations are impacting visitor conversions. Thereport is customizable and can show how visitors responded during avisit as well as across repeated visits to show if a variation affectedthe likelihood of a conversion on a return visit. Reports can includecharts and graphs with statistical summaries supporting the results. Themodification module provides the ability to track, correlate, andmeasure visitor actions on a web server to one or more test groups.

Features and Advantages

One advantage of the invention is that when integrating with webanalytics platforms, the invention saves the time and effort that wouldotherwise be required to manually insert or modify the tags across pagesand templates.

Another advantage is that website operators can try new third-party webanalytics platforms without manually modifying all web page tags on agiven website.

A feature of the invention is that there is no need to modify the webserver. Website operators can create site variations using a WYSIWYGeditor. The advantage of a visual editor is saving time and costotherwise spent on web designers and programmers.

Another feature of the invention is that the website operator cansegment visitors by specific criteria. The advantage of criteria-basedsegmentation is being able to target optimization efforts to the highestvalue market segment(s). The invention enables segmenting with severaltypes of criteria, including historical visitor data stored within theinvention as well as that provided by the web operator's externalCRM/SFA (Customer Relationship Management/Sales Force Automation)system.

Another feature of the invention is that variations can be created forboth statically and dynamically created content. Variations are createdusing a search and replace technique involving “regular expressions.”Use of regular expressions enables more complex variations to beintroduced, particularly on dynamically-generated content.

Another feature of the invention is multivariate/matrix testing. Thismethod of testing achieves higher response rates in less time than A/Btesting, by testing any number of variations for any number of factorsanywhere on the web server, alone or in combination.

Another feature of the invention is that it can provide built-inreports. The advantage of reports, with accompanying charts and graphs,is that it gives web operators conclusive answers, with statisticalsummaries to support the results.

Another feature of the invention is sophisticated test campaignscheduling. The modification module lets a website operator run testcampaigns individually or concurrently. Also, test campaigns can bescheduled in advance, and run on an automated basis where the tests stoponce certain conditions have been met.

Another feature of the invention is reusable test campaign elements.Website operators save time by creating new test campaigns usingelements from previous test campaigns. The modification module's librarystores previous variations, response points, and assignment testcriteria for subsequent use.

Another feature of the invention is the ability to track and analyzevisitor actions. The system can track not only individual actions, butalso sequences of actions such as specific click-paths through ashopping visit. The modification module can even track how variationsaffected actions across visits. The modification module can trackrequests for HTML content as well as non-HTML content.

Another feature of the invention is the ability to signal to the webserver that a current request came from a visitor in a specific testgroup. This enables website operators to test functional variations inthe web server by creating code that can alter its function based on thevisitor's test group status.

A feature of the reverse proxy model is that the invention canoptionally cache static content, e.g. images, in the reverse proxyserver for web acceleration, and can increase speed of network trafficby optionally deflating or compressing content as it is sent back in theHTTP response.

An advantage of the invention is that the website operator does not needto modify website content, or require special programs on the visitor'sweb browser. The system can track visitor actions with or withoutcookies.

Another advantage of the invention is that the general system cost ismoderate. With no website modification needed, website operators canavoid reconfiguring web server architecture. No needed websitemodification also means a short campaign set-up time.

Another advantage of the invention is compatibility with all webcontent. The system works on all sites, static or dynamic content, andeven sites spread across multiple hosts.

By combining the proven methods of direct response testing andexperimental design, the invention enables website operators to: 1)determine which changes improve conversion rates by testing the changeson a subset of visitors, 2) test unlimited variations using a web-basedmanagement system that requires no technical expertise or modificationsto the web server, 3) optimize the web server for specific customergroups using visitor segmentation, 4) seamlessly integrate withthird-party web analytics platforms for accessing web site variationsdata.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, closely related figures and items have the same numberbut different alphabetic suffixes. Processes, states, statuses, anddatabases are named for their respective functions.

FIG. 1 is an overall system diagram showing the reverse proxy servermodel.

FIG. 2 is a system diagram showing hardware of the reverse proxy servermodel.

FIG. 3 is a flow chart of entry point response.

FIG. 4 is a flow chart of user identification and user management.

FIG. 5 is a flow chart for content management, including response pointanalysis and variation transformation.

DETAILED DESCRIPTION OF THE INVENTION, INCLUDING THE PREFERREDEMBODIMENT

Operation

In the following detailed description of the invention, reference ismade to the accompanying drawings which form a part hereof, and in whichare shown, by way of illustration, specific embodiments in which theinvention may be practiced. It is to be understood that otherembodiments may be used, and structural changes may be made withoutdeparting from the scope of the present invention.

FIG. 1 is an overall system diagram showing the reverse proxy servermodel. The model begins after a web page request originates (1) from webbrowser 100. Instead of sending the request directly to target server130 (that is the address where a request is originally targeted, or theintended web server of a given request), the request is sent to reverseproxy server 120, over Internet 110, or over some other network. Reverseproxy server 120 includes a proxy module 122 for handling allproxy-related tasks and a modification module 124 for modifying content,tracking visitors actions, collecting data for selected visitor responsepoints, and reporting on how content changes affect conversion rates.Data related to test campaigns, visitors, users, and content variationsis stored in one or more databases (not shown), which can be implementedas part of reverse proxy server 120 or elsewhere.

Continuing with FIG. 1, reverse proxy server 120 requests (2) contentfrom target server 130. Target server 130 responds to reverse proxyserver 120 by sending (3) unmodified content to reverse proxy server120. Reverse proxy server 120 tracks a visitor's session using a cookieof reverse proxy server 120 or target server 130. Reverse proxy server120 may also track the visitor by URL/path translation via a piggy-backapproach. Reverse proxy server 120 evaluates the request and responseand tracks the visitor's actions. Reverse proxy server 120 modifies webcontent, sent by target server 130, to introduce variations as required.Finally reverse proxy server 120 responds to web browser 100 by sending(4) modified content.

FIG. 2 is a system diagram showing hardware of the reverse proxy servermodel. FIG. 2 illustrates a plurality of client computers 300 runningweb browsers 100 (not shown). Reverse proxy computer 320 runs reverseproxy server 120, and target computer 330 runs target server 130. Clientcomputer 300, reverse proxy computer 320, and target computer 330 areinterconnected and communicate via network 310, which is the Internetfor the Application Service Provider (ASP) model (where the applicationis hosted on the service provider's computer) or a LAN for theenterprise model (where the application is hosted on the client'scomputer). It is to be understood that reverse proxy computer 320 andtarget computer 330 can be one or more physical computers. Reverse proxycomputer 320 can be any computer operable to run a database server and aweb server.

The system is able to compensate for reverse proxy server 120 failure.If reverse proxy server 120 is down, target server 130 can still beavailable by deploying a hot backup server (not shown) with anintelligent load balancer. The load balancer transparently redirectstraffic to the hot backup. Another option is to deploy edge-routingservices to redirect traffic from one connectivity network to anothernetwork. For example, the system can be collocated at a data center thathas peering with four tier-one Internet backbones (not shown). If apacket that would ordinarily travel through (for example) backbone 1cannot be transmitted, then the edge-router (not shown) detects this andre-routes the packet, and subsequent packets, through (for example)backbone 2. Another option is to deploy DNS failover, whereby if the DNSserver detects that the reverse proxy server is down, then subsequentDNS queries return the IP address of the target computer. When the DNSserver detects that the reverse proxy server has returned to normaloperation, subsequent DNS queries return the IP address of the reverseproxy server.

The system is also able to compensate for any delay with HTTP requestsand responses between reverse proxy server 120 and target server 130 byclosely positioning the servers, or placing the servers on the samenetwork in a data center. Caching static content, and deflating andcompressing content also compensates for any delays.

When a website operator launches a test campaign, the website operatortells the system, via a user interface (not shown) to the modificationmodule 124 of reverse proxy server 120, exactly which type(s) ofvisitors to test. For example, if a target server 130 sells to a broadmarket, then perhaps the operator of target server 130 will simply wanta representative sample of all visitors, such as 5% of all visitors overa two week period of time. If a business targets one or more specificmarket segments, then the system lets the business target effortsprecisely. The system provides several types of segmentation criteria,including past visitor history, referrals from search engines or bannerads, entry through specific landing pages, and customer segments definedby the web operator's Customer Relationship Management/Sales ForceAutomation (CRM/SFA) system. The number and types of the segmentationcriteria is adjustable according to the needs of a particular targetserver.

Generally, the system will vary or modify HTML content, since themajority of web pages appear in this format, but the system is notlimited to varying HTML content. The system can vary content for plaintext documents, WML documents, CSS documents, and other non-HTMLcontent.

Embodied as a web software product, the invention can be offered as anASP version (application service provider), offered as an enterprise(installed) version, and other formats. In one embodiment of theinvention (ASP model), the software runs as a hosted service on a webserver in a third-party data center or facility. For very high trafficsites (enterprise model), a standalone version can be installed in awebsite operator's own data center. The system is operable to runmultiple concurrent test campaigns with the ability to signal avisitor's test status through the use of a cookie or other HTTP headertransmitted between the reverse proxy server 120 and target server 130for functionality testing.

FIG. 3 is a flow chart of entry point response. In step 320 the reverseproxy server 120 (not shown) receives an HTTP request from a clientcomputer 300 (not shown). The target website operator decides attributesof the HTTP request that indicate whether or not the request will resultin modified or unmodified content being delivered to the client computer300. In step 335, if the visitor's HTTP request includes attributes(such as IP address, USER-AGENT URI path, or content-type) that thetarget website operator has not defined for tracking, then the HTTPrequest will be passed through, resulting, in step 330, in no variationof the target website's content and no tracking of the visitor'srequest, and then operation proceeds to step 335, where response contentis transmitted to the visitor. Otherwise, operation proceeds to step340. In step 340, additional protocol checks are performed to determinewhether tracking and variation should be performed. For example, if therequest does not produce an “OK” response code (e.g. HTTP status codenumber 200) from the target web server, then the request wilt not betracked. If the protocol checks of step 340 indicate that no trackingshould be done, then operation proceeds to step 330. Otherwise,operation proceeds to step 345. In step 345, the system checks to see ifthe HTTP request is for text-based content, and, if so, then content isbuffered in step 350. If not, operation proceeds to step 355. In step355, anti-caching is performed (response headers are adjusted) toprevent caching by downstream proxies and browsers. Next, operationproceeds to step 360, user identification and management, which isdescribed in FIG. 4.

FIG. 4 is a flow chart of user identification and user management. Instep 400, a visitor is identified via cookie data or URI (UniformResource Identifier) data contained in the visitor's request. Operationproceeds to step 405, where the system checks to see if the visitor hasa Globally Unique Identifier (GUID). If the result of step 405 is no,operation proceeds to step 430, and if the result of step 405 is yes,operation proceeds to step 410. In step 410, the system looks up theuser and retrieves the most recent user visit data, and operationproceeds to step 415. In step 415, the system checks to see if user datawas found, and if no, then operation proceeds to step 430. If yes, thenoperation proceeds to step 420. In step 420 the system checks to see ifany user visit data was found for the user, and if no, then operationproceeds to step 430. If yes, then operation proceeds to step 425. Instep 425, the system checks to see if the user's most recent user visitdata has expired, and if yes, then operation proceeds to step 445. Ifno, then operation proceeds to step 498. A new page tag is eitherinserted, where a page tag does not exist, or transformed, where thereis an existing page tag. The process then continues to contentmanagement in step 499.

Continuing with FIG. 4, in step 430, the system checks to see whetherthe user is being tracked, and if no, then operation proceeds to step435. In step 435, the system performs a pass-through response, i.e., novariation of the target website's content and no tracking of thevisitor's request, and operation proceeds to step 436, where responsecontent is transmitted to the visitor. If the result of step 430 is yes,then operation proceeds to step 440, where the system creates a userrecord and, if a reverse proxy server is managing session data, then thesystem also sets a cookie. Then operation proceeds to step 470.

Continuing with FIG. 4, in step 445 the system checks to see if the userwas previously assigned to a test campaign, and if no, then operationproceeds to step 470. If yes, operation proceeds to step 450. In step450, the system looks up visit and time limits for the assigned testcampaign, and operation proceeds to step 455. In step 455, the systemchecks to see whether the user was previously assigned to a testcampaign, and if no, then operation proceeds to step 470. If yes, thenoperation proceeds to step 455. In step 455, the system checks to seewhether the user still qualifies for assignment to the test campaign,and if yes, then operation proceeds to step 497. If no, operationproceeds to step 460. In step 460, the system updates user data toremove the current assignment and inserts user assignment history data,and then operation proceeds to step 465. In step 465, the system checksto see whether the visitor can be reassigned to another test campaign,and if yes, then operation proceeds to step 470. If no, then operationproceeds to step 495. In step 495, the system checks to see whether thisuser visit should be logged, and if yes, then operation proceeds to step497. If no, then operation proceeds to step 498. A new page tag iseither inserted, where a page tag does not exist, or transformed, wherethere is an existing page tag. Operation, then, proceeds to step 499,continue to content management.

Continuing with FIG. 4, in step 470, the system evaluates the availabletest campaigns and user assignment criteria. A user can be included orexcluded from a given test campaign based on a plurality ofoperator-defined variables, including, but not limited to, the visitor'sand/or request's date/time, the USER AGENT value, IP address, entry URL,current URL, referring URL, visitor history, cookie value, and pastvisitor actions. Next, operation proceeds to step 480. In step 480, thesystem checks to see whether the user qualifies for a new test campaignassignment, and if no, then operation proceeds to step 495. If yes, thenoperation proceeds to step 485. In step 485, the system assigns the userto a test campaign and variation group, and operation proceeds to step495. In step 490, the system stores the assigned test campaign (TC) andvariation group (VG) information in user data, and operation proceeds tostep 497.

Continuing with FIG. 4, in step 497, a new user visit is created. Instep 498 a new page tag is either inserted, where a page tag does notexist, or transformed, where there is an existing page tag. Operationproceeds to step 499, continue to content management, which is describedin FIG. 5.

FIG. 5 is a flow chart for content management, including response pointanalysis and variation transformation. Operation begins at step 500,where the system checks to see whether a user is assigned to a testcampaign, and if no, then operation proceeds to step 540. If yes,operation proceeds to step 505, where the system loops through responsepoints for the current test campaign, and then operation proceeds tostep 510. In step 510, the system checks to see whether there were anyresponse point hits, and if no, then operation proceeds to step 520. Ifyes, then operation proceeds to step 515, where the system recordshit(s) for the user visit, dynamically modifies any third-party webanalytics tags, URL's, forms or cookies to insert new tracking codes,and, if required, updates the hit rating for the user visit. Operation,then, continues to step 517 where test campaign values are injected intoweb pages tags. In step 520, the system checks to see whether the useris assigned to a non-control variation group, and if no, then operationproceeds to step 540. If yes, then operation proceeds to step 525, wherethe system performs variations for the current variation group. Next,operation proceeds to step 520. In step 530, the system checks to seewhether any variations were applied, and if no, then operation proceedsto step 540. If yes then operation proceeds to step 535, where thesystem records hit(s) for the user visit and, if required, updates thehit rating for the user visit. Operation, then, continues to step 537where test campaign values are injected into web pages tags beforeproceeding to step 540. In step 540, the system updates data, includinguser visit data, page view count data (only for non-image requests) andtime-stamp data, and then operation proceeds to step 545, where responsecontent is transmitted to the visitor.

The invention preferably integrates test campaign values into JavaScriptweb page tagging by either inserting values, creating new tags, ortransforming tags. Below is an example of a WebSideStory JavaScript webpage tag without test campaign data included:

<script language=“javascript”> var _hbEC=0,_hbE=new Array;function_hbEvent(a,b){b=_hbE[_hbEC++]=new Object( );b._N=a;b._C=0;return b;} varhbx=_hbEvent(“pv”);hbx.vpc=“HBX0200u”;hbx.gn=“ehg-examplewebsite.hitbox.com”; //BEGIN EDITABLE SECTION //CONFIGURATIONVARIABLES hbx.acct=“WATTS000EJH00000”;//ACCOUNT NUMBER(S)hbx.pn=“PUT+PAGE+NAME+HERE”;//PAGE NAME(S)hbx.mlc=“CONTENT+CATEGORY”;//MULTI-LEVEL CONTENT CATEGORYhbx.pndef=“title”;//DEFAULT PAGE NAME hbx.ctdef=“full”;//DEFAULT CONTENTCATEGORY //OPTIONAL PAGE VARIABLES //ACTION SETTINGS hbx.fv=“”;//FORMVALIDATION MINIMUM ELEMENTS OR SUBMIT FUNCTION NAME hbx.lt=“none”;//LINKTRACKING hbx.dlf=“n”;//DOWNLOAD FILTER hbx.dft=“n”;//DOWNLOAD FILENAMING hbx.elf=“n”;//EXIT LINK FILTER //SEGMENTS AND FUNNELShbx.seg=“”;//VISITOR SEGMENTATION hbx.fnl=“”;//FUNNELS //CAMPAIGNShbx.cmp=“”;//CAMPAIGN ID hbx.cmpn=“”;//CAMPAIGN ID IN QUERYhbx.dcmp=“”;//DYNAMIC CAMPAIGN ID hbx.dcmpn=“”;//DYNAMIC CAMPAIGN ID INQUERY hbx.dcmpe=“”;//DYNAMIC CAMPAIGN EXPIRATION hbx.dcmpre=“”;//DYNAMICCAMPAIGN RESPONSE EXPIRATION hbx.hra=“”;//RESPONSE ATTRIBUTEhbx.hqsr=“”;//RESPONSE ATTRIBUTE IN REFERRAL QUERYhbx.hqsp=“”;//RESPONSE ATTRIBUTE IN QUERY hbx.hlt=“”;//LEAD TRACKINGhbx.hla=“”;//LEAD ATTRIBUTE hbx.gp=“”;//CAMPAIGN GOALhbx.gpn=“”;//CAMPAIGN GOAL IN QUERY hbx.hcn=“”;//CONVERSION ATTRIBUTEhbx.hcv=“”;//CONVERSION VALUE hbx.cp=“null”;//LEGACY CAMPAIGNhbx.cpd=“”;//CAMPAIGN DOMAIN //CUSTOM VARIABLES hbx.ci=“”; hbx.hc1=“”;hbx.hc2=“”; hbx.hc3=“”; hbx.hc4=“”; hbx.hrf=“”;//CUSTOM REFERRERhbx.pec=“”;//ERROR CODES //INSERT CUSTOM EVENTS //END EDITABLE SECTION//REQUIRED SECTION. CHANGE “YOURSERVER” TO VALID LOCATION ON YOUR WEBSERVER (HTTPS IF FROM SECURE SERVER) </script><scriptlanguage=“javascript1.1”src=“http://www.examplewebsite.com/hbx.js”></script>

Below is an example of a WebSideStory JavaScript web page tag with testcampaign data inserted:

<script language=“javascript”> var _hbEC=0,_hbE=new Array;function_hbEvent(a,b){b=_hbE[_hbEC++]=new Object( );b._N=a;b._C=0;return b;} varhbx=_hbEvent(“pv”);hbx.vpc=“HBX0200u”;hbx.gn=“ehg-examplewebsite.hitbox.com”; //BEGIN EDITABLE SECTION //CONFIGURATIONVARIABLES hbx.acct=“WATTS000EJH00000”;//ACCOUNT NUMBER(S)hbx.pn=“PUT+PAGE+NAME+HERE”;//PAGE NAME(S)hbx.mlc=“CONTENT+CATEGORY”;//MULTI-LEVEL CONTENT CATEGORYhbx.pndef=“title”;//DEFAULT PAGE NAME hbx.ctdef=“full”;//DEFAULT CONTENTCATEGORY //OPTIONAL PAGE VARIABLES //ACTION SETTINGS hbx.fv=“”;//FORMVALIDATION MINIMUM ELEMENTS OR SUBMIT FUNCTION NAME hbx.lt=“none”;//LINKTRACKING hbx.dlf=“n”;//DOWNLOAD FILTER hbx.dft=“n”;//DOWNLOAD FILENAMING hbx.elf=“n”;//EXIT LINK FILTER //SEGMENTS AND FUNNELShbx.seg=“”;//VISITOR SEGMENTATION hbx.fnl=“”;//FUNNELS //CAMPAIGNShbx.cmp=“”;//CAMPAIGN ID hbx.cmpn=“”;//CAMPAIGN ID IN QUERYhbx.dcmp=“”;//DYNAMIC CAMPAIGN ID hbx.dcmpn=“”;//DYNAMIC CAMPAIGN ID INQUERY hbx.dcmpe=“”;//DYNAMIC CAMPAIGN EXPIRATION hbx.dcmpre=“”;//DYNAMICCAMPAIGN RESPONSE EXPIRATION hbx.hra=“”;//RESPONSE ATTRIBUTEhbx.hqsr=“”;//RESPONSE ATTRIBUTE IN REFERRAL QUERYhbx.hqsp=“”;//RESPONSE ATTRIBUTE IN QUERY hbx.hlt=“”;//LEAD TRACKINGhbx.hla=“”;//LEAD ATTRIBUTE hbx.gp=“”;//CAMPAIGN GOALhbx.gpn=“”;//CAMPAIGN GOAL IN QUERY hbx.hcn=“”;//CONVERSION ATTRIBUTEhbx.hcv=“”;//CONVERSION VALUE hbx.cp=“null”;//LEGACY CAMPAIGNhbx.cpd=“”;//CAMPAIGN DOMAIN //CUSTOM VARIABLES hbx.ci=“12345678”; //INJECTED VALUE: customer IDhbx.hc1=“Example_Test_Campaign_(——)Jan_1_(——)2005”; // INJECTED VALUE:test campaign name hbx.hc2=“5678”; // INJECTED VALUE: variation group ID(“recipe ID”) hbx.hc3=“Homepage_Headline|Version1,Homepage_Subhead|Version3,Homepage_Image|Version9”; // INJECTED VALUE: list of 3 testfactors & variations hbx.hc4=“”; hbx.hrf=“”;//CUSTOM REFERRERhbx.pec=“”;//ERROR CODES //INSERT CUSTOM EVENTS //END EDITABLE SECTION//REQUIRED SECTION. CHANGE “YOURSERVER” TO VALID LOCATION ON YOUR WEBSERVER (HTTPS IF FROM SECURE SERVER) </script><scriptlanguage=“javascript1.1”src=“http://www.examplewebsite.com/hbx.js”></script>

Other Embodiments

In another embodiment, a separate third-party reverse proxy server isused. The functions of the proxy module are performed by the third-partyreverse proxy server, and the modification module is a stand-alonesoftware product that operates in conjunction with the third-partyreverse proxy server. In this embodiment the modification module acceptsinput from and transmits output to the reverse proxy server, and nosoftware changes to the reverse proxy server or the target server arerequired.

In another embodiment, the modification module is a stand-atone softwareproduct. In this embodiment, the reverse proxy server is optional. Thetarget server receives a request from a web browser for web content fromthe target website hosted at a target server; and the modificationmodule retrieves unmodified web content from the target server, modifiesthe web content (including performing a search and replace operationusing regular expressions), delivers the modified web content to therequesting web browser, tracks actions of the visitor, collects data foreach response point selected to be measured, and reports on how contentchanges affect conversion rates. In this embodiment, both theApplication Service Provider (ASP) model (where the modification moduleis hosted on the service provider's computer) or a LAN for theenterprise model (where the modification module is hosted on theclient's computer) can be used. A variation of the ASP model involvesimplementing the modification module in a computer “appliance” thatcould be installed in the client's premises or in the service provider'spremises.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reviewing the abovedescription. The scope of the invention should, therefore be determinedwith reference to the appended claims, along with the full scope ofequivalents to which such claims are entitled.

The invention claimed is:
 1. A computerized method of testing variationsof website content, the method comprising: receiving a request at areverse proxy server for website content located at a web server; saidreverse proxy server retrieving requested website content from the webserver, wherein the website content contains a web page tag associatedwith a web analytics platform; modifying said website content, at thereverse proxy server, by introducing variations in the website contentthat is visible to a website user; inserting values associated withwebsite content variations into the web page tag, whereby data fromwebsite content variations is accessible within a reporting system of aweb analytics platform; and transmitting modified website content to arequestor; wherein the visible website content variations are introducedto a subset of all website users.
 2. The method of claim 1, wherein theweb page tag associated with the web analytics platform uses JavaScript.3. The method of claim 1, wherein the web analytics platform is a thirdparty platform.
 4. The method of claim 1, wherein modifying said webcontent includes introducing multiple variations into a single web page.5. A computerized method of testing variations of website content, themethod comprising: receiving a request at a reverse proxy server forwebsite content located at a web server; said reverse proxy serverretrieving requested website content from the web server; modifying saidwebsite content at the reverse proxy server by introducing variations inthe website content that is visible to a website user; inserting a webpage tag in the modified website content, wherein the web page tag isformatted for use with a web analytics platform; inserting valuesassociated with website content variations into the web page tag,whereby data from website content variations is accessible within areporting system of the web analytics platform; and transmittingmodified website content to a requestor; wherein the visible websitecontent variations are introduced to a subset of all website users. 6.The method of claim 5, wherein the web page tag associated with the webanalytics platform uses JavaScript.
 7. The method of claim 5, whereinthe web analytics platform is a third party platform.
 8. The method ofclaim 5, wherein website content is static or dynamic.
 9. The method ofclaim 5, further comprising providing a visual editor for varyingwebsite content.
 10. A computerized method of testing variations ofwebsite content, the method comprising: receiving a request at a reverseproxy server for website content located at a web server; said reverseproxy server retrieving requested website content from the web server,wherein the website content contains a web page tag formatted for usewith a first web analytics platform; modifying said website content atthe reverse proxy server by introducing variations in the websitecontent that is visible to a website user; modifying web page taggingwithin the requested website content, wherein modified web page taggingis formatted for use with a second web analytics platform; insertingvalues associated with website content variations into the web page tag,whereby data from website content variations is accessible within areporting system of the second web analytics platform; and transmittingmodified website content to a requestor; wherein the visible websitecontent variations are introduced to a subset of all website users. 11.The method of claim 10, wherein modifying the web page tagging includesreplacing an existing web page tag.
 12. The method of claim 10, whereinmodifying the web page tagging includes appending a second web page tag.13. The method of claim 10, wherein the web page tag associated with theweb analytics platform uses JavaScript.
 14. A system for testingvariations of website content, the system comprising: a reverse proxyserver operable to: receive a request for website content located at aweb server; retrieve requested website content from the web server;modify said website content by introducing variations in the websitecontent that is visible to a website user; insert values associated withwebsite content variations into web page tagging within the websitecontent, whereby data from website content variations is accessiblewithin a reporting system of a web analytics platform; and transmitmodified website content to a requestor; wherein the visible websitecontent variations are introduced to a subset of all website users. 15.The system of claim 14, wherein the web page tagging includes web pagetags integrated with the website content located at the web server. 16.The system of claim 14, wherein the reverse proxy server is furtheroperable to insert a web page tag into modified website content.
 17. Thesystem of claim 14, wherein the reverse proxy server is further operableto replace, within the website content, a first existing web page tagthat is associated with a first web analytics platform, with a secondweb page tag that is associated with a second web analytics platform.18. The system of claim 14, wherein the reverse proxy server is furtheroperable to append, within the website content, a second web page tagthat is associated with a second web analytics platform, to websitecontent containing an existing web page tag that is associated with afirst web analytics platform.
 19. The system of claim 14, wherein theweb page tagging uses JavaScript.